ABSTRACT
BackgroundVaccines play a crucial role in the response to COVID-19 and their efficacy is thus of great importance.AimTo assess the robustness of COVID-19 vaccine efficacy (VE) trial results using the fragility index (FI) and fragility quotient (FQ) methodology.MethodsWe conducted a Cochrane and PRISMA-compliant systematic review and meta-analysis of COVID-19 VE trials published worldwide until 22 January 2023. We calculated the FI and FQ for all included studies and assessed their associations with selected trial characteristics using Wilcoxon rank sum tests and Kruskal-Wallis H tests. Spearman correlation coefficients and scatter plots were used to quantify the strength of correlation of FIs and FQs with trial characteristics.ResultsOf 6,032 screened records, we included 40 trials with 54 primary outcomes, comprising 909,404 participants with a median sample size per outcome of 13,993 (interquartile range (IQR): 8,534-25,519). The median FI and FQ was 62 (IQR: 22-123) and 0.50% (IQR: 0.24-0.92), respectively. FIs were positively associated with sample size (p < 0.001), and FQs were positively associated with type of blinding (p = 0.023). The Spearman correlation coefficient for FI with sample size was moderately strong (0.607), and weakly positive for FI and FQ with VE (0.138 and 0.161, respectively).ConclusionsThis was the largest study on trial robustness to date. Robustness of COVID-19 VE trials increased with sample size and varied considerably across several other important trial characteristics. The FI and FQ are valuable complementary parameters for the interpretation of trial results and should be reported alongside established trial outcome measures.
Subject(s)
COVID-19 Vaccines , COVID-19 , Humans , COVID-19/prevention & control , Randomized Controlled Trials as TopicABSTRACT
Purpose: Significant results of randomized controlled trials (RCTs) should be properly weighed. This study adopted fragility index (FI) to evaluate the robustness of significant dichotomous outcomes from RCTs on coronavirus disease 2019 (COVID-19) treatment. Materials and methods: ClinicalTrials.gov and PubMed were searched from inception to July 31, 2021. FIs were calculated and their distribution was depicted. FI's categorical influential factors were analyzed. Spearman correlation coefficient (r s) was reported for the relationship between FI and the continuous characteristics of RCTs. Results: Fifty RCTs with 120 outcomes in 7869 patients were included. The FI distribution was abnormal with median 3 (interquartile range 1-7, P = 0.0001). The FIs and robustness were affected by the outcomes of interest, various patient populations, and interventions (T = 18.215,16.667, 23.107; P = 0.02,0.0001, 0.001, respectively). A cubic relationship between the FIs and absolute difference of events between groups with R square of 0.848 (T = 215.828, P = 0.0001, R square = 0.865) was observed. A strong negative logarithmic relationship existed between FI and the P value with R square = - 0.834. Conclusion: The robustness of significant dichotomous outcomes of COVID-19 treatments was fragile and affected by the outcomes of interest, patients, interventions, P value, and absolute difference of events between the groups. FI was an useful quantitative metric for the binary significant outcomes on COVID-19 treatments. Registration: PROSPERO (CRD42021272455). Supplementary Information: The online version contains supplementary material available at 10.1007/s44231-022-00027-y.
ABSTRACT
BACKGROUND: The effectiveness of corticosteroids in acute respiratory distress syndrome (ARDS) and COVID-19 still remains uncertain. Since ARDS is due to a hyperinflammatory response to a direct injury, we decided to perform a meta-analysis and an evaluation of robustness of randomised clinical trials (RCTs) investigating the impact of corticosteroids on mortality in ARDS in both COVID-19 and non-COVID-19 patients. We conducted a systematic search of the literature from inception up to 30 October 2020, using the MEDLINE database and the PubMed interface. We evaluated the fragility index (FI) of the included RCTs using a two-by-two contingency table and the p-value produced by the Fisher exact test; the fragility quotient (FQ) was calculated by dividing the FI score by the total sample size of the trial. RESULTS: Thirteen RCTs were included in the analysis; five of them were conducted in COVID-19 ARDS, including 7692 patients, while 8 RCTS were performed in non-COVID ARDS with 1091 patients evaluated. Three out of eight RCTs in ARDS had a FI > 0 while 2 RCTs out of five in COVID-19 had FI > 0. The median of FI for ARDS was 0.625 (0.47) while the median of FQ was 0.03 (0.014). The median of FI for COVID-19 was 6 (2) while the median of FQ was 0.059 (0.055). In this systematic review, we found that FI and FQ of RCTs evaluating the use of corticosteroids in ARDS and COVID-19 were low.